Dynamical Automata 0.1 the Chomsky Hierarchy

The recent work on automata whose variables and parameters are real numbers has focused largely on questions about computational complexity and tractability. It is also revealing to examine the metric relations that such systems induce on automata via the natural metrics on their parameter spaces. This brings the theory of computational classiication closer to theories of learning and statistical modeling which depend on measuring distances between models. With this in mind, I develop a generalized method of identifying pushdown automata in one class of real-valued automata. I show how the real-valued automata can be implemented in neural networks. I then explore the metric organization of these automata in a basic example, showing how it eshes out the skeletal structure of the Chomsky Hierarchy and indicates new approaches to problems in language learning and language typology. 1 0. Introduction Smolensky (1990) argues that connectionist (or \neural") networks ooer an opportunity to overcome the brittleness of symbolic devices without foregoing their powerful computational capabilities. \Brittleness" refers to the fact that many symbolic devices are catastrophically sensitive to small distortions in their encoding|a bit or a semicolon out of place can bring an entire system to its knees. Such sensitivity is reminisicent of the trademark behavior of \chaotic" dynamical processes: small diierences in initial conditions give rise to substantial diierences in long-term behavior. It would be ironic, then, if the interpretation of neural devices as dynamical systems with potentially chaotic behaviors led to a realization of Smolensky's ideal. Intriguingly, this is the character of the result that I report on here. Fractal objects, which turn up as the traces of chaotic processes, turn out to be especially useful for instantiating powerful computing devices in systems of neurons which exhibit graceful modiication under small distortions (cf. Pollack, 1991). It is as though by embracing the caprice of a chaotic process, a computational system can stay in its good graces and make eeective use of its complexity (cf. I work here with the now-standard notion of what it means for a system to be computa-tionally powerful or complex. The Chomsky Hierarchy is an ordering of formal languages into increasingly more-inclusive classes. A formal language, L, is taken to be a set of strings of symbols. A computer which can be made to output any string of symbols in L but no string of symbols in the complement of L is called a generator for L. …